Analysis of Software Project Reports for Defect Prediction Using KNN

Defect severity assessment is highly essential for the software practitioners so that they can focus their attention and resources on the defects having a higher priority than the other defects. This would directly impact resource allocation and planning of subsequent defect fixing activities. In this paper, we intend to predict a model which will be used to assign a severity level to each of the defect found during testing. The model is based on text mining and machine learning technique. We have used KNN machine learning method to predict the model employed on an open source NASA dataset available in the PITS database. Area Under the Curve (AUC) obtained from Receiver Operating Characteristics (ROC) analysis is used as the performance measure to validate and analyze the results. The obtained results show that the performance of KNN technique is exceptionally well in predicting the defects corresponding to top 100 words for all the severity levels. Its performance is less for top 5 words, better for top 25 words and still better for top 50 words. Hence, with these results, it is reasonable to claim that the performance of KNN is dependent on the number of words selected as independent features. As the number of words increases, the performance of KNN also gets better. Apart from this, it has been noted that KNN method works best for medium severity defects as compared to the other severity defects.